gh-121647: Prefer decltype() for _Py_TYPEOF() on C++ - #157267

!buildbot AMD64 Windows11 Non-Debug 3.x PR

The regex 'AMD64 Windows11 Non-Debug 3.x PR' did not match any buildbot builder. Is the requested builder in the list of stable builders?

Right now, I don't access to a Windows machine to test my change, so I will rely on Windows buildbots for now.

!buildbot AMD64 Windows11 Non-Debug 3.x

The regex 'AMD64 Windows11 Non-Debug 3.x' did not match any buildbot builder. Is the requested builder in the list of stable builders?

!buildbot AMD64 Windows11 Non-Debug

🤖 New build scheduled with the buildbot fleet by @chris-eibl for commit 85a6ba9 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157267%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 Windows11 Non-Debug

The builders matched are:

• AMD64 Windows11 Non-Debug PR

Oh, test_cppext failed on "Ubuntu / build and test (ubuntu-26.04-arm)" CI:

ERROR: test_build (test.test_cppext.TestInteralCAPI.test_build)
(...)
  /usr/bin/g++ -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -fno-strict-overflow -Wsign-compare -g -Og -Wall -fPIC -I/home/runner/work/cpython/cpython-builddir/build/test_python_61202æ/tempcwd/env/include -I/home/runner/work/cpython/cpython-ro-srcdir/Include -I/home/runner/work/cpython/cpython-builddir -c extension.cpp -o build/temp.linux-aarch64-cpython-316-pydebug/extension.o -Werror -DMODULE_NAME=_testcppext_internal -DTEST_INTERNAL_C_API=1
  In file included from /home/runner/work/cpython/cpython-ro-srcdir/Include/object.h:742,
                   from /home/runner/work/cpython/cpython-ro-srcdir/Include/Python.h:79,
                   from extension.cpp:13:
  /home/runner/work/cpython/cpython-ro-srcdir/Include/internal/pycore_object.h: In function ‘void _PyObject_XSetRefDelayed(PyObject**, PyObject*)’:
Error:   /home/runner/work/cpython/cpython-ro-srcdir/Include/cpython/object.h:377:26: error: cannot declare pointer to ‘PyObject*&’ {aka ‘struct _object*&’}
    377 |         _Py_TYPEOF(dst)* _tmp_dst_ptr = &(dst); \
        |                          ^~~~~~~~~~~~
  /home/runner/work/cpython/cpython-ro-srcdir/Include/internal/pycore_object.h:757:5: note: in expansion of macro ‘Py_XSETREF’
    757 |     Py_XSETREF(*p_obj, obj);
        |     ^~~~~~~~~~
  error: command '/usr/bin/g++' failed with exit code 1

test.pythoninfo says CC.version: gcc (Ubuntu 15.2.0-16ubuntu1) 15.2.0.

buildbot/AMD64 Windows11 Non-Debug PR

Sadly, it still fails with the same error. I suppose that MSVC doesn't target C++11 by default.

test_build (test.test_cppext.TestInteralCAPI.test_build) ... Processing b:\uildarea\pull_request.ware-win11.nondebug\build\Lib\test\wheeldata\setuptools-79.0.1-py3-none-any.whl

  extension.cpp(299): error C3861: '__typeof__': identifier not found

Even internal__testcpp14ext which uses /std:c++14 also fails:

extension.cpp(299): error C3861: '__typeof__': identifier not found

By the way, test_limited_cpp03ext logs a compiler warning:

cl : Command line warning D9002 : ignoring unknown option '/std:c++03'

I suppose that MSVC doesn't target C++11 by default.

Without /Zc:__cplusplus MSVC always sets __cplusplus=199711L.

Detect whether the /std option is in effect during a C++ compilation with the _MSVC_LANG preprocessor macro. Because some existing code depends on the value of the macro __cplusplus being 199711L, the MSVC compiler doesn't change the value of this macro unless you explicitly opt in by setting /Zc:__cplusplus. Specify /Zc:__cplusplus and the /std option to set __cplusplus to the appropriate value.

Maybe we can use the _MSVC_LANG preprocessor macro, because many people do not use /Zc:__cplusplus?

cl : Command line warning D9002 : ignoring unknown option '/std:c++03'

Yupp, that's not a valid option per https://learn.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=msvc-170. It "starts" with /std:c++14.

The Microsoft C++ compiler in Visual Studio 2017 and later versions doesn't support C++ standards modes earlier than C++14 (/std:c++14). Such support isn't planned.

But again, without /Zc:__cplusplus we won't see anything else than __cplusplus=199711L and would need to query _MSVC_LANG.

!buildbot AMD64 Windows11 Non-Debug PR

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 13de0a0 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157267%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 Windows11 Non-Debug PR

The builders matched are:

• AMD64 Windows11 Non-Debug PR

But again, without /Zc:__cplusplus we won't see anything else than __cplusplus=199711L and would need to query _MSVC_LANG.

Ok, let me retry with _MSVC_LANG macro :-)

For the Ubuntu failure, I can reproduce it locally on Fedora 44 (GCC 16.2.1) with the C program:

struct object {
    int data;
};

typedef struct object PyObject;

#define Py_CLEAR(obj) \
    do { \
        decltype(obj)* ptr = &obj; \
        *ptr = nullptr; \
    } while (0)


void func(PyObject **obj)
{
    Py_CLEAR(*obj);
}

int main()
{
    PyObject *obj = nullptr;
    func(&obj);
}

It fails to build with:

x.cpp: In function ‘void func(PyObject**)’:
x.cpp:9:24: error: cannot declare pointer to ‘PyObject*&’ {aka ‘struct object*&’}
    9 |         decltype(obj)* ptr = &obj; \
      |                        ^~~
x.cpp:16:5: note: in expansion of macro ‘Py_CLEAR’
   16 |     Py_CLEAR(*obj);
      |     ^~~~~~~~

The fix is to replace decltype(obj)* ptr with decltype(&(obj)) ptr. C++ is more strict about types, especially when using & in the type declaration.

Since _MSVC_LANG is always set when __cplusplus is defined, I've just successfully tried

#elif defined(__cplusplus) && (__cplusplus >= 201103L ||  _MSVC_LANG >= 201103L)

locally.

Given that MSVC doesn't support anything older than /std:c++14, I'd also mark the other tests like

    @unittest.skipIf(support.MS_WINDOWS, "MSVC doesn't support /std:c++11")
    def test_build_cpp11(self):

!buildbot AMD64 Windows11 Non-Debug PR

🤖 New build scheduled with the buildbot fleet by @chris-eibl for commit ae56459 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157267%2Fmerge

The command will test the builders whose names match following regular expression: AMD64 Windows11 Non-Debug PR

The builders matched are:

• AMD64 Windows11 Non-Debug PR

The fix is to replace decltype(obj)* ptr with decltype(&(obj)) ptr.

I pushed a change for that.

Ah good, test_cppext passed on "Ubuntu / build and test (ubuntu-26.04-arm)" CI.

Aha, test_cppext also passed on the Windows buildbot.

!buildbot Windows

🤖 New build scheduled with the buildbot fleet by @vstinner for commit dec5373 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F157267%2Fmerge

The command will test the builders whose names match following regular expression: Windows

The builders matched are:

• AMD64 Windows PGO Tailcall PR
• AMD64 Windows11 Non-Debug PR
• AMD64 Windows Server 2025 Refleaks PR
• AMD64 Windows Server 2025 Clang PR
• AMD64 Windows Server 2022 NoGIL PR
• AMD64 Windows PGO NoGIL Tailcall PR
• ARM64 Windows PR
• ARM64 Windows Non-Debug PR
• AMD64 Windows10 PR
• AMD64 Windows PGO PR
• AMD64 Windows PGO NoGIL PR